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DETAILED ACTION 
Drawings 

1 . The drawings were received on January 1 3, 2003. These drawings are objected: 
The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 

because they do not include the following reference sign(s) mentioned in the 
description: 

Figure 1 the box labeled as "2 Image Positioner" does not have a reference 
numeral. 

Figure 5A does not have the "a" label as mention in the specification of page 18, 
line 19. 

Figure 6A has a reference numeral "80b" listed twice. 

A proposed drawing correction or corrected drawings are required in reply to the 
Office action to avoid abandonment of the application. The objection to the drawings 
will not be held in abeyance. 

Specification 

2. The disclosure is objected to because of the following informalities: 

Page 8, lines 3,10,16 are referring to numeral 60 of figure 1. However, numeral 
60 of figure 1 is a "3D Object Incorporator" and not a "two-image positioner" as 
mentioned with numeral 60 of page 8. A suggestion is to assign a different number for 
the "two-image positioner" of figure 1 . 

Page 8, line 19: "by" should be deleted. 

Page 9, line 11: The second "relative" of line 1 1 should be deleted. 
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Page 14, line 5: insert "be" between "to" and "corrected". 

Pages 19-21 have numerous errors such as page 19, line 6: "(206)" should be 
replaced with "(1206)", and "(FIG. 4F)" should be changed to "(FIG. 6F)". And a better 
correspondence between figure 6F and the specification at page 19,20 is required: The 
specification is referring to reference numerals of figure 6F which are not numbered 
properly. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-14, 17-21, 23-38, 41-45, 47, and 48 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Kumar et al. (US Patent 5,963,664 A) and in view of 
Hsu et al. (US Patent 6,078,701 A). 

Regarding claims 25-48, Kumar teaches that the image processing system is 
used on a computer that is programmed at col. 4, lines 59-60. 

Regarding claims 1 and 25, Kumar et al. discloses a method comprising: 
determining an orientation (Figure 3 is a geometric representation at Kumar 
et al.: col. 5, line 63) of a camera (Kumar et al.: fig. 3, label "O" and "M" are focal 
centers of each camera at col. 6, lines 3,4.) associated with a first image (Kumar et al.: 
fig. 3, num. 304) based on a shape (Kumar aligns images using regions as a shape at 
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col. 7 line 64 thru col. 8, line 2) of a corrected version (Using figure 3,the image, num. 
304 or first image is warped to num. 302 or the reference image at col. 6, lines 6-9) of 
the first image, wherein the corrected version of the first image has less perspective 
distortion (Warping is performed to align images 302 and 304 at col. 6, lines 6-9) 
relative to a reference image (fig. 3, num. 302) than the first image and the shape of the 
corrected version of the first image is different (The first image is warped) from the 
shape of the first image; and 

projecting the first image on a surface (Images 302 and 304 are aligned to 
surface S of figure 3, numeral 300) based on the orientation of the camera 
associated with the first image. 

Kumar et al. does not teach the use of a perimeter as a shape. 

However, Hsu et al. does teach a use of perimeter for aligning images at col. 1 1 , 
line 11. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use the perimeter teaching of Hsu et al. instead of using the regions of 
Kumar et al. because "In practice, only a subset of points is selected. Indeed, the local 
registration technique might only determine... [a mapping]... for a sparse set of 
points, not the entire overlap between images i and j, for reasons of efficiency or lack of 
image features. Even if... [a mapping]... were known everywhere, it is neither efficient 
nor necessary to sum over all points (Hsu et al. at col. 1 1 , lines 1-6)." 

Regarding claims 2 and 26, Kumar et al. teaches the method of claim 1 further 
comprising: 
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determining a focal length (The focal length of a camera is predetermined 
at col. 4, lines 51-53) of a camera associated with the first image based on the shape of 
the perimeter of the corrected version of the first images, wherein the step of projecting 
the first image is further based on the focal length (Kumar et al. uses perspective 
projection with a focal length at col. 5, lines 52-62.). 

Claims 3 and 27 were addressed in claim 1 above. 

Regarding claims 4 and 28, Kumar et al. teaches the method of claim 3 further 
comprising: 

merging the projected reference image and the projected first image to 
form a panoramic image (Kumar et al. uses a plurality of images that are 
combined to form a panoramic image at col. 3,lines 61-65). 

Regarding claims 5 and 29, Kumar et al. teaches the method of claim 3 further 
comprising: 

projecting a three-dimensional object onto the surface (Kumar states, 
"More particularly, the parallax field represents the objects that lie in front of and behind 
the parametric surface and the distance (height) of these objects from the surface, i.e., 
the three-dimensional geometry of the scene (col. 4, lines 45-49)."); 

merging the projected three-dimensional object, the reference image and 
the first image to form a panoramic image (The "P-then-P" method initially registers two 
images then determines the said parallax field at col. 4, lines 6-10.). 

Regarding claims 6-8 and 30-32, Kumar et al. teaches using a parametric 
surface at col. 4, line 2. 
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Regarding claims 9 and 33, which is similar to claim one except for requiring a 
second image. Kumar et al. teaches the method of claim 9 as shown in claim 1 above of 
a plurality of images as shown in fig. 2, numerals 200. 

Regarding claimslO and 34, Kumar et al. teaches the method of claim 2 wherein 

determining the focal length and rotation angle further comprises: 

selecting initial values ("initial parameter values" at col. 8, line 35 are shown in 

col. 7, equation "(5)" and are part of the "Sequential Registration" process at col. 6, line 
58) for the orientation and the focal length (The parameters values include 
orientation "Q" and focal length T as shown in equation 5.); and 

improving the accuracy (The sequential registration process uses a "coarse-to- 
fine, iterative manner" at col. 7, line 64,65) of the selected values of the orientation and 
the focal length by: 

estimating the shape (Initially, the entirety of the images are selected as the 
regions of interest which are used for registering images at col. 8, lines 1-6) of the 
corrected version of the first image based on the selected values of the orientation and 
the focal length; 

comparing (fig. 4, numeral 414 is an "iterate" query step) the estimated shape 
and the actual shape (The actual shape is based on a minimizing error function that is 
calculated for each region for alignment at col. 8 , lines 6-10.) 

adjusting the selected values of the orientation and the focal length based 
on a difference (Equation 8 at column 8 is a sum of squared difference (SSD) measure 
used for the selected regions or estimated shape) between the estimated shape and the 
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actual shape of the perimeter of the corrected version of the first image. 

Regarding claims 1 1 and 35, Kumar et al. teaches the method of claim 10 

wherein improving the accuracy of the selected values of the orientation and the focal 

length further comprises: 

computing a difference between the selected values of the orientation and 

the focal length with the adjusted values of the orientation and the focal length; 
if the computed difference (SSD) is below a threshold value ("predefined 

threshold SSD level" at col. 8, line 48): 

determining that the adjusted values of the orientation and the adjusted value of 
the focal length are the actual orientation and the actual focal length (Kumar et al. 
states," The routine queries, at step 414, whether further computational iterations are 
necessary to achieve alignment (col. 8, lines 45-47).") ; 

otherwise, if the computed difference is not below the threshold value: 

selecting the adjusted values of the orientation and the focal length as the values 
of the orientation and the focal length (A current set of parameters are used for the next 
iteration at col. 8, lines 38-42); and 

repeating ("iterively refines the parameters" at col. 8, line 36) the step of 
improving the accuracy of the selected values of the orientation and the focal length. 

Regarding claims 12 and 36, Kumar et al. teaches the method of claim 10 
wherein the initial value of the orientation is selected to be an orientation of a camera 
associated with the reference image (The eight motion parameters (col. 7, equation 5) 
contain translational views "T" in three directions between cameras are used in 
determining the initial orientation parameters at col. 7 , lines 38,39 and col. 8, line 35). 
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Claims 1 3 and 37 were addressed in claim 1 above. 

Regarding claims 14 and 38, Kumar et al teaches the method of claim 10 
wherein the initial value of the focal length is selected based on a measurement of the 
first image (Using figure 4, Kumar et al states,"... the routine estimates, in a coarse-to- 
fine manner, the motion parameters that align the two images to one another... (col. 8, 
lines 23-25)." Note that the motion parameters include a focal length.). 

Regarding claims 1 7 and 41 , Kumar et al. teaches the method of claim 1 wherein 
the orientation comprises a rotation angle of the camera (Kumar et al. teaches an 
angular-velocity at col. 7, line 39). 

Claims 18 and 42 are similar to claim one except for requiring an overlap of the 
first and reference image. Kumar et al. teaches overlapping of at least two images at 
col. 2, lines 25-28. Kumar et al. also uses selected regions of interest which are 
overlapping regions at col. 8, lines 42-44. 

Regarding claims19 and 43, Kumar et al. teaches the method of claim 18 further 
comprising: 

determining a position offset ("residual displacement" at col. 8 , line 41) of the 
first segment of the view relative to the reference segment of the view, wherein 
correcting ("alignment iteration" at col. 8, line 39) for perspective distortion is based on 
the determined position offset. Therefore, a residual displacement or offset is 
determined between two images through alignment iteration or correcting. 
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Regarding claims 20 and 44, Kumar et al. teaches the method of claim 18 
wherein the first image includes at least a first reference point and a second reference 
point (Kumar et al. uses regions and each region contains pixels for each image at col. 
8, line 33) correcting for perspective distortion alters the shape of the first image 
("transformation... is applied to the inspection image... (Kumar et al. at col.8, lines 39- 
41).") by moving the first reference point relative to the second reference point (The 
image regions are matched over selected regions of the images at col. 7, line 66 thru 
col.8, line 2.) 

Kumar et al does not teach the use of a perimeter; however Hsu et al. does teach 
a use of a perimeter as addressed in claim 1 above. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use a perimeter of Hsu et al. instead of a region as taught by Kumar et 
al. for the same reason as claim 1 above. 

Regarding claims 21 and 45, Hsu et al. teaches the first and second reference 
points are vertices ("corner points" at Hsu et al., col. 11, line 11) defined by the shape of 
the perimeter of the first image. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use corner points of Hsu et al. instead of a region as taught by Kumar 
et al. for the same reason as claim 1 above. 

Regarding claims 23 and 47, Kumar et al. teaches the method of claim 1 wherein 
determining the orientation ("angular velocity" or "Q" is a variable used to estimate eight 
motion parameters as shown in column 7, equation 5 at col. 8, lines 30-33) is further 
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based on the shape ("reference to a region" at col. 8, line 32) of the first image. 

Kumar et al does not teach the use of a perimeter; however Hsu et al. does teach 
a use of a perimeter as addressed in claim 1 above. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use a perimeter of Hsu et al instead of a region as taught by Kumar et 
al. for the same reason as claim 1 above. 

Regarding claim 24 and 48, Kumar et al. teaches the method of claim 1 wherein 
the perimeter of the first image has the same shape as the perimeter of the reference 
image (Kumar states,"... a single camera [can] produce a series of video frames). 
Therefore each frame, which includes a perimeter, is taken from the same camera. 
5. Claims 15 and 39 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kumar et al. (US Patent 5,963,664 A) and in view of Hsu et al. (US Patent 
6,078,701 A) and further in view of Yevick (US Patent 4,057,338 A). 

Regarding claims 15 and 39, Kumar et al teaches the method of claim 14 
wherein the selected initial value of the focal length is predetermined, but does not 
teach the remaining portion of claims 15 and 39. 

However, Yevick does teach a focal length is the sum of a length and width of the 
image. Yevick states that the "focal length is approximately the length of the diagonal of 
the film (col. 10, lines 10,11)". Therefore, using geometry a diagonal of an image can be 
determined using a sum of the squares of the lengths of the edges of the image. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to determine the focal length of camera because Yevick teaches a 
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"known... "rule-of-thumb"..." method that a diagonal of an image is an approximate of 
the focal length of a camera (col. 10, lines 8-11). 

6. Claims 16 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kumar et al. (US Patent 5,963,664 A) and in view of Hsu et al. (US Patent 
6,078,701 A) and further in view of Yamada et al. (US Patent 5,802,202 A). 

Regarding claims 16 and 40, Kumar et al. teaches the method of claim 10 
wherein an iteration ("iterively refines the parameters" at col. 8, line 36) is used to adjust 
the initial values of the rotation angle and the focal length (The eight motion parameters 
include a focal length T and angular-velocity "Q" as shown in col. 7, equation 5). 

Kumar et al. does not teach using Newton's iteration; however Yamada, in the 
analogous field of three dimensional imaging, teaches the use of the "Newton-Raphson 
method" for solving for a set of parameters to transform an object to a standard 
coordinate system at col. 29, lines 23-48. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use the Newton-Raphson method as taught by Yamada to modify the 
iteration as taught by Kumar et al. because the Newton-Raphson method quickly 
deterimines the position of an object for transforming coordinates even when the whole 
object is difficult to detect at Yamada, col. 29, lines 48-50. 

7. Claims 22 and 46 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kumar et al. (US Patent 5,963,664 A) and in view of Hsu et al. (US Patent 
6,078,701 A) and further in view of Hirosawa et al. (US Patent 5,838,837). 
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Regarding claim 22 and 46, Kumar teaches a shape of the first image and 
correcting for perspective distortion alters ("warp" or "transformation" at Kumar, col. 6, 
line 6 and col. 8, line 39) the shape ("region" at Kumar, col. 8, line 32) of the first image 
as addressed in claim 1 , but does not teach the use of a rectangle or trapezoid as a 
shape. 

However, Hirosawa et al. does teach, in the analogous field of image synthesis, 
teaches the use of a rectangle (fig. 16A, numerals 84,85,86) and trapezoid (fig. 16B, 
numerals 84,85\86') for synthesizing images at col. 13, lines 1-45). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use the synthesis teaching as taught by Hirosawa et al. as the shape or 
region as taught by Kumar et al. because the deformed images reduce clipping of 
images due to a limited memory space at col. 13, lines 1-7. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Herman et al. (US Patent 6,075,905 A) is pertinent as teaching a method 
of transforming all images except for a reference image to form a mosaic at col. 15, 
lines 15-19. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario-Vasquez whose telephone number is 
703-305-5431 . The examiner can normally be reached on 9-5. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on 703-305-4706. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Dennis Rosario-Vasquez 
Unit 2621 
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